Approaches for Incorporating a Variety of Metadata in Transformer Operation

A plain transformer model typically leverages only one piece of metadata - position encoding - directly in the transformer model. The use of transformers typically involves expensive and complex external scaffolding before or after output generation to avoid issues such as hallucination, irrelevance, etc. This disclosure describes techniques to incorporate a variety of metadata types into the native architecture of transformer models. The additional signals can help avoid hallucinations, improve relevance, and minimize the need of expensive external scaffolding. Generalizing transformer operation to incorporate a diversity of metadata can be achieved in various ways such as adding a metadata embedding layer, conditioning self-attention on the metadata, conditioning with gated self-attention, employing a different encoder-decoder architecture, etc. Different types of metadata can help in different ways to improve the quality of the output generated by the transformer and reduce hallucinations. The techniques described in this disclosure can also support multimodal data, such as images, audio, video, or text, etc., with the metadata used representing the specific mode

Power generation from waste of IC engines

Several methods for waste thermal energy recovery from internal combustion engine (ICE) have been studied by using supercharger or turbocharger and /or combined. This study presents an innovative approach on power generation from waste of IC engine based on coolant and exhaust. The waste energy harvesting system of coolant (weHSc) is used to supply hot air at temperatures in the range of 60–70 C directly into the engine cylinder, which would be useful to vaporize the fuel into the cylinder. The waste energy harvesting system of exhaust system (weHSex) has been developed with integrating fuzzy intelligent controlled Micro-Faucet emission gas recirculation (MiF-EGR) and thermoelectric generator (TEG). In this study the MiF-EGR (micro-facet exhaust gas recirculation) will be used to maintain the intake temperature 70 C by keeping flow of the exhaust to the engine cylinder chamber and to increase the engine volumetric efficiency. The TEG produces electrical power from heat flow across a temperature gradient of exhaust and delivers DC electrical power to the vehicle electrical system which could reduce the load of the alternator by as much as 10%. The performance of weHS equipped engine has been investigated by using GT suite software for optimum engine speed of 4000 rpm. The result shows that specific fuel consumption of engine has improved by 3% due to reduction of HC formation into the engine combustion chamber causes significantly improved the emission. While, the brake power has been increased by 7% due to the fuel atomization and vaporization at engine intake temperature 70 C

Rethinking Attention with Performers

We introduce Performers, Transformer architectures which can estimate regular (softmax) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness. To approximate softmax attention-kernels, Performers use a novel Fast Attention Via positive Orthogonal Random features approach (FAVOR+), which may be of independent interest for scalable kernel methods. FAVOR+ can be also used to efficiently model kernelizable attention mechanisms beyond softmax. This representational power is crucial to accurately compare softmax with other kernels for the first time on large-scale tasks, beyond the reach of regular Transformers, and investigate optimal attention-kernels. Performers are linear architectures fully compatible with regular Transformers and with strong theoretical guarantees: unbiased or nearly-unbiased estimation of the attention matrix, uniform convergence and low estimation variance. We tested Performers on a rich set of tasks stretching from pixel-prediction through text models to protein sequence modeling. We demonstrate competitive results with other examined efficient sparse and dense attention methods, showcasing effectiveness of the novel attention-learning paradigm leveraged by Performers.Comment: Published as a conference paper + oral presentation at ICLR 2021. 38 pages. See https://github.com/google-research/google-research/tree/master/protein_lm for protein language model code, and https://github.com/google-research/google-research/tree/master/performer for Performer code. See https://ai.googleblog.com/2020/10/rethinking-attention-with-performers.html for Google AI Blo

Impact of COVID-19 pandemic on the physical, mental and social health of the suburban and rural adult population in Bangladesh

Background and objectives: The COVID-19 pandemic caused a significant impact on health worldwide. Adverse effect of COVID-19 on health-related quality of life is significant. This study aimed to find out the impact of COVID-19 on the physical, mental and social health of suburban and rural adult population in Bangladesh. Methods: A suburban and a rural community were purposively selected. The suburban and rural areas were located about 40 km and 130 km north and north-east of Dhaka city respectively. People aged ≥20 years in the selected communities were enrolled in the study. The investigation procedure included socio-demographic and clinical history, anthropometry, and clinical examination and laboratory investigations. Depression, Anxiety and Stress Scale-21 (DASS-21) and 36-Item Short Form Health Survey (SF-36) questionnaires were used for assessing mental and social health respectively. Knowledge, attitude and practice (KAP) regarding the prevention and transmission of COVID-19 was assessed by a validated questionnaire and interview. Results: Total 385 individuals (suburban=201, rural=184) were enrolled in the study. Out of 385, 116 and 269 were male and female, respectively. Out of total 385 participants, depression, anxiety and stress were present in 113 (29.4%), 144 (37.4%) and 70 (18.2%) respectively, while 210 (54.5%) were normal. Extremely severe depression, anxiety and stress were present in 3.6%, 6% and 0.5%, respectively. Depression and anxiety did not differ between suburban and rural populations, though stress was significantly higher among the suburban (p<0.05) population. Social functioning was limited in more than 50% as opposed to excellent (5.5%) or good (39.8%). Almost 60% of the participants had to cut-down schedule of heavy work. Moderate to minimal physical activities were less affected, though weakness and nervousness predominantly hindered socialization. About the prevention and transmission of COVID-19, awareness and attitude were found satisfactory (≥45%), though practice was neglected (<30%). Conclusions: This is the first study in Bangladesh to report the impact of the COVID-19 pandemic on the physical, mental, and social health of adult suburban and rural populations. Physical and mental disabilities were evident among the studied people. Social functioning was affected by COVID-19 equally in suburban and rural participants. A well-designed cohort study is needed to obtain a real picture of the impact of COVID-19 pandemic on human health and society. IMC J Med Sci. 2024; 18(1):007. DOI: https://doi.org/10.55010/imcjms.18.007 *Correspondence: MA Sayeed, Department of Community Medicine and Public Health, Ibrahim Medical College, 1/A Ibrahim Sarani, Segunbagicha, Dhaka 1000, Bangladesh; Email: [email protected]